

```{r setup, include = FALSE}
# This line of code tells the document all the display defaults
knitr::opts_chunk$set(echo = TRUE, include = TRUE, fig.align="center",warnings = FALSE,fig.width = 5,fig.height=3)
```

# Background Information

We have discussed hypothesis testing for a simple linear regression.  We have also discussed $r^2$ and how it is used to determine the percent of variation in $Y$ that can be explained by its *linear* relationship with $X$.  Now, we will show a new way to obtain this information that can be easily extended to multiple linear regression.

**Analysis of Variance (ANOVA)**: A table that breaks down the sources of the variation in the response variable, $Y$, when we include the explanatory variable, $X$, in our linear model

$$\text{Total Sums of Squares} = \text{SST} = \sum_{i=1}^n (y_i - \bar{y})^2$$ $$ = \sum_{i=1}^n (\hat{y}_i - \bar{y})^2 + \sum_{i=1}^n (y_i - \hat{y}_i)^2 $$ $$ = \text{Model Sums of Squares} + \text{Sum of Squared Errors}$$
where $\hat{y}_i = \hat{\beta}_0 + \hat{\beta}_1x_i$.

**Model Sums of Squares (SSM)**:  The amount of variation in $Y$ explained by the linear model with $X$

$$\text{SSM} = \sum_{i=1}^n (\hat{y}_i - \bar{y})^2$$

**Error Sums of Squares (SSM)**:  The amount of variation in $Y$ *not* explained by the linear model with $X$
$$\text{SSE} = \sum_{i=1}^n (y_i - \hat{y}_i)^2$$
**Mean Squared for Model (MSM)**:  The variance of $Y$ explained by the linear model with $X$

$$\text{MSM} = \frac{\text{SSM}}{p-1},$$
where $p - 1$ is the *model degrees of freedom* for our linear model, and $p$ is the number of parameters ($\beta$s) in our linear model.

- You can also think of the *model degrees of freedom* as the number of $\beta$s we add to the reduced model to get the full model.

**Mean Squared Error (MSE)**:  The variance of $Y$ not explained by the linear model with $X$

$$\text{MSE} = \frac{\text{SSE}}{n - p},$$
where $n - p$ is the *error degrees of freedom* for our linear model.

- For simple linear regression, the *error degrees of freedom* are $n-2$.

**F Statistic**: A statistic that measures the ratio of the variances of the response variable $Y$ that is explained/not explained by the model, $$F = \frac{\text{MSM}}{\text{MSE}} = \frac{\text{Explained Variance}}{\text{Unexplained Variance}}$$

- For Simple linear regression, $$F = \frac{\hat{\beta}_1^2}{s_{\hat{\beta}_1}^2} = \left(\frac{\hat{\beta}_1}{s_{\hat{\beta}_1}}\right)^2 = t^2$$

Recall from Chapter 3, if $t \sim t_{df=\nu}$, then $F = t^2 \sim F_{df1=1,df2=\nu}$.  Then, we can equivalent say that, if we are testing for a significant linear relationship between $X$ and $Y$, the p-value would be calculated as $$p-value = P(F_{df1=1,df2=\nu} > F) = P\left(F_{df1=1,df2=\nu} > \frac{\text{MSM}}{\text{MSE}}\right).$$

- If $p-value \leq \alpha$, then we reject $H_0$ and say there is a significant linear relationship between $X$ and $Y$.

- If $p-value > \alpha$, then we fail to reject $H_0$ and say there is not a significant linear relationship between $X$ and $Y$.

## ANOVA Table Format

|     |df    |Sums of Squares  |Mean Square  |F Value  |Pr(>f)   |
|-----|:-----|:----------------|:------------|:--------|:--------|
|Model|$p-1$ | $\text{SSM}$             | $\text{MSM} = \frac{\text{SSM}}{p-1}$| $F=\frac{\text{MSM}}{\text{MSE}}$ | $P(F_{df1=p-1,df2=n-p} > F)$ |
|Error|$n-p$ | $\text{SSE}$              | $\text{MSE} = \frac{\text{SSE}}{n-p}$|         |         |
|Total|$n-1$ | $\text{SST} = \text{SSM} + \text{SSE}$ |             |         |         |   

## Example

Use the following incomplete ANOVA table to answer the following questions.

|     |df    |Sums of Squares  |Mean Square  |F Value  |Pr(>f)   |
|-----|:-----|:----------------|:------------|:--------|:--------|
|Model|1 | 50.83             | |  |  |
|Error|48|  | |         |         |
|Total| | 98.48 |             |         |         |   


\begin{tabular}{lrrrrr}
  \hline
& df & Sums of Squares & Mean Square & f Value & Pr(>f) \\
  \hline
  Model & 1 & 50.83 & & & \\ 
  Error & 48 & & &  &  \\ 
     \hline
  Total & & 98.48 &&&
\end{tabular}

a. What is the total degrees of freedom?

\vspace{2cm}

b. What is the sum of squared errors?

\vspace{2cm}

c. What is the mean squares of the model and the mean squared error?

\vspace{2cm}

d. What is the F statistic and p-value for testing for a significant linear relationship?

\vspace{2cm}

## Example

Using the bloodpressure dataset, obtain an ANOVA table and perform a hypothesis test for a significant linear relationship.

```{r}
```

## What can we obtain from an ANOVA table?

We can find an estimate for the regression variance/mean squared error.  $$s^2 = \text{MSE} = \frac{\text{SSE}}{n-2}$$

We can find the proportion of variance explained by the model ($r^2$).
$$r^2 = \frac{\text{SSM}}{\text{SST}}.$$

